The amount of data traffic over large networks, such as the Internet, continues to increase. Such increase is the result of continued exponential increases in the number of users, as well as in the number of large documents, such as media files, to which these users desire access. As a result of this unprecedented demand in the need for bandwidth and access to networks, Internet Service Providers (ISPs), backbone providers, and other carriers that provide the physical connections which implement the Internet face correspondingly unprecedented difficulty.
The Internet provides widespread access to content on an equal basis through the use of a client and server communication model. In this structure, certain computers known as “servers” are used to store and provide information. One type of server, known as a host server, provides access to information such as data, text, documents, programs stored in various computer file formats, but generally referred to as a “document.” Other computers in the network known as “clients” allow the users to view documents through the use of a computer program known as a browser that requests a copy of the document be sent from host servers down to the client. Documents are typically requested by the client browser program specifying an address which identifies the host server which stores the document. The request is sent over the network to a naming service in order to obtain instructions for how to establish a connection with the host server associated with the address. Once this connection is established, the server retrieves the document from its local disk and transmits the document over network to the client. The connection between the client and host server is then terminated. A given request may require that it pass through a number of routers or “hops” through the Internet on its way from the host server down to the client.
A common solution for the present bottlenecks within the Internet is to deploy higher speed interconnection hardware. Such solutions include the deployment of digital subscriber line (XDSL) and cable modem technology to speed up the access between the end users and points of presence. Gigabit speed routers and optical fiber backbones are also being proposed to alleviate congestion within the network itself. At the server site, server clusters and load balancers are being deployed to assist with the dispatching of Web pages more efficiently.
While all of these solutions provide some expediency, each only solves part of the problem, and none provides a satisfactory solution to the ultimate problem—the path between the client and server is only as fast or as slow as the slowest link.
As it turns out, much of the traffic on the Internet is redundant in the sense that different users request the same documents from the same servers over and over again. Therefore, it is becoming increasingly apparent that certain techniques, such as distributed content caching, may be deployed to reduce the demand for access to both the servers and to the network routing infrastructure. Distributing content throughout the network, such as through the use of document caches, provides a way to intercept client requests and serve copies of the original document to multiple client locations.
Using a cache, the process for providing document files to the client computers changes from the normal process. In particular, when the client requests the connection, say to a given server, the intermediate cache server may instead be requested to obtain the document. While the document is being transmitted down to the client computer, a copy is stored at the intermediate cache server. Therefore, when another client computer connected to the same network path requests the same content as the first user, rather than requiring the request to travel all the way back to the host server, the request may be served from the local cache server.
By moving popular content closer to the users who want such content, distributed content servers may be used to alleviate the congestion at its cause. These distributed cache servers dramatically improve end user response time, decrease backbone and server loading, provide a vehicle for efficient routing of time sensitive traffic.
This approach to caching may not prove useful for current content servers. In particular, current content servers often use more complicated Uniform Resource Locator (URL) schemes for specifying content access paths. Some servers map incoming URL requests to the URLs of the content that is finally delivered. For example, the access URL may contain a session key that is used for computing the final resource to be accessed. Such mapping poses severe problems on any content caching mechanism. Specifically, because of this mapping, the same access URL may lead to the delivery of different resources. In addition, different access URLs may result in the delivery of the same resource. Therefore, the primary access URL cannot be used by a cache server for identifying resources.